Evaluating the Results of Methods for Computing Semantic Relatedness

The semantic relatedness between two concepts is a measure that quantifies the extent to which two concepts are semantically related. Due to the growing interest of researchers in areas such as Semantic Web, Information Retrieval and NLP, various approaches have been proposed in the literature for automatically computing the semantic relatedness. However, despite the growing number of proposed approaches, there are still significant criticalities in evaluating the results returned by different semantic relatedness methods. The limitations of the state of the art evaluation mechanisms prevent an effective evaluation and several works in the literature emphasize that the exploited approaches are rather inconsistent. In this paper we describe the limitations of the mechanisms used for evaluating the results of semantic relatedness methods. By taking into account these limitations, we propose a new methodology and new resources for comparing in an effective way different semantic relatedness